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Lesson 3 

DEMONSTRATION OF INDEXING 



Review 

Before we attempt to index a research document, let us review 
the procedure discussed in the last lesson. 

Concept In d exing . Recognition of the significant concepts in a 
document is called ’’concept indexing. ” It has nothing to do with the 
particular ’’index language” to be used. 

What is your first step in concept indexing? 

Looking for concepts in the body 

of tne document. 4 



Finding key words in the title of 
the document. 



6 



2 



You have arrived at this page by mistake. 

Return to page 1 and follow the page directions beside your 
answer choice. 



3 

(from pages 5, 6) 



After you have compiled the list of candidate terms, you feel that 
you must look up the terms in the ERIC Thesaurus (ERICT). 

Right. The purpose of ERICT is to control the vocabulary of the 
indexer and the searcher. 

Translation of Candidate Terms into Controlled Index Terms . After 
you have compiled your list of candidate terms, you must translate each 
term into the language of ERICT. 

1. Is your term used as a descriptor in ERICT? If so, use it. 

2. Does ERICT list your term but refer to a synonym? If so, use 
use ERICT” s preference. 

3. Does ERICT combine two or more of your terms? If so, use the 
combined term. 

4. Does ERICT indicate that you should use a related term or add- 
itional terms? If so, do it. 

What happens if you can^t find one of your candidate terms in 
ERICT? 

That term must not be used. 1C 

You must recommend the term for 
inclusion in the thesaurus. 



(from page 1) 



You say that your first step in concept indexing would be to 
look for concepts in the body of the document. 

That comes later. You should start with the title. 

Please continue on 6, 












5 

(from page 6) 



After you have compiled the list of candidate terms, you feel that 
you must discard the less important terms. 

No. You should decide whether a concept is needed in the indexing 
before you list its candidate term. 



Please continue on 3. 



6 

(from p?ges 1 and 4) 

You say that your first step in concept indexing would be finding 
key words in the title of the document. 

Yes. This is the most likely place for a description of what the 
document is all about . In the title you will usually find some of your key 
words — words that will stand (alone, or in combination with others) for 
a class of documents . 

You can test a term by asking the question: "Have you any information 

onx?" 

For a document entitled "Reading Readiness — How to determine it. " 
You could ask: 

Have you any information on how ? 

Have you any information on reading ? 

Have you any information on readiness ? 

Have you anything on determine or determining? 

Obviously, the first question is useless. In the others, the words substituted 
for "x" have meaning, either by themselves or with another term, such as 
"reading readiness" or "determining readiness. " They are therefore possible 
keywords. 

After you have compiled a list of all the candidate terms, what do you 
do with them? 

Look up each term in the 

ERIC Thesaurus. 3 

Discard the less important 

terms. 5 



o 

ERIC 



7 

(from pages 3, 10) 

Yes. K a concept is significant, and if there is no term for it in 
ERICT, you must make a recommendation for an addition. 

The Office of Education has published a guide to the selection of 
terms with the title ”Rules for thesaurus preparation. ” You will also 
find in use at your Clearing House a special form for justifying the term 
of your choice. The procedure for completing this will be explained by 
your supervisor. 

We are now ready to look at the demonstration document, ”The 
Use of 'LiKability' Ratings and Ability Scores in the Prediction of School 
Achievement.” (J. Robert Williams. The Journal of Educational Research , 
Vol. 57, No. 2: pp. 90-92; October, 1963) Please read this document 
carefully and then turn to 8. 



8 

(from page 7) 



Demonstration document 



When making your list of candidate terms you will find it an aid to 
clear thinking if you record them in columns which separate different 
categories. We show on this page a model including examples of terms 
for each category* You can try it out and modify it, if necessary. Once a 
satisfactory outline has been discovered, blanks can be duplicated for 
regular use at the Center. 



The column for kind of document has been included, even though it 
is not really part of subject indexing, to give you some practice in distinguish- 
ing between form words and subject words. 



1 


2 


3 


4 


5 


6 


7 


8 


K5nH of 


Educational 

Establishment 


Person 


Attributes 
or Actions 
(of terms 
in any 


Curric- 


Teaching 

and 

Learning 
Methods 
and Admin- 


Testing, 

Evalu- 

ation 

Measure- 


Agent or 
Means 


Docutnent 




Studied 


column^ 


ulum 


istration 


ment 


of Col, 7 


e.g. 


e.g. 


e.g. 


e.g. 


e.g. 


e.g. 


e.g. 


e.g. 


Educa. 


School 


Teacher 


Attitudes 


Math. 


Lectures 


Rating--' 


Teacher 


Experi- 

ment 


High School 


Student 


Objective 


Physics 


Television 


Grading-' 


Objective 


Case 

Study 

Survey 

Educa- 
Re search 


University 

Classroom 


Doctor 






Planning 

Discipline 


Comparin 


test 

5 



Make sure that you have a copy of this form ready and enter each 
candidate term as we proceed with the lesson. 

Please refer to Appendix E of Volume 1 for sample of complete 
form for co^^ying. 

Please continue on 9. 



9 

(from page 8) 

Before looking for subject words we must ask if the document as 
a whole needs a form description. We suggested in Lesson 2 that the 
major categories for ERIC documents are: 

E3q>eriments 
Case studies 
Surveys 

Research review 

To which category would you allocate this article? 



E3q)eriments 


11 


Case studies 


15 


Surveys 


17 


Research review 


19 



None of these 



21 



10 

(from page 3) 



K you can^t find one of your candidate terms in ERICT, you think 
that it must not be used. 

No. The most likely reason for a term^s absence is that it has 
not previously been used for indexing. Part of your job as an ERIC 
indexer is to recommend new terms as required. 



Please continue on 7. 



11 

(from page 9) 

Yes. The section headed "Purpose and procedure" indicates 
that this article belongs to the category of Experiments. 

We can now examine the title of the article for keywords. 

The Use Of Likability Ratings And Ability Scores 
In The Prediction of School Achievement 

Which list of candidate terms would you select? 

Likability, Ratings, Ability, 

Scores, Prediction of school 
achievement. 



Likability ratings. Ability scores. 
School achievement. Achievement 
prediction. 



12 



12 



(from pages 11, 14) 

Yes. Whenever you have very general terms, it is best to list 
them as candidates for pre- coordination. 

1. Likability Ratings 

2. Ability Scores 

3. School Achievement 

4. Achievement Prediction 

They should be recorded on your form in the following manner (we 
are adding the form word in column 1): 



1 




5 



Teaching 

and 

Learning 



6 



7 




Kind of 



Attributes 
or Actions 
(of terms 
in any 



& Admin- Evaluation 



Methods Testing, 



Document 




column) 



Curriculum istration Measurement 



Educ, 

ixperiment 



Likability- - 

Ability •• 

School 

Achievemei t 



Ratings 

Scores 



Achievemei t 



Prediction 



Please turn to 13 



13 

(from page 12) 



We can now begin looking for terms in the text. 

AN EARLIER article (4) reported the development of 
a method for obtaining teacher ratings of students on 
the trait of ’’likability" and presented results showing 
that these ratings were reasonably reliable and signifi- 
cantly correlated with both ability scores and grade- 
point averages in a sample of high- school seniors. On 
the basis of these findings and the known relationship 
of ability to achievement, the possibility of combining 
’’likability" ratings and ability scores was suggested as 
a means of improviiig achievement prediction. 

In terms of the data from the preliminary study, the 
expectation that employing such a combination of 
measures offered better prediction than using either 
alone came from the fact that, whereas their separate 
correlations with grade-point averages were .550 and 
. 668 respectively, their combined (multiple) correlation 
with this criterion was . 728. The standard score form 
of the multiple regression or predictive formula in this 
instance , as developed previously , was 

= .307z 2 . 54Oz0, (1) 

where z^ Zg, and Zg represented grade point average, 

ability score, and "likability" rating, in that order, for 
any student. 

In the first paragraph, which term denotes the genus of the species 
"likability"? 

Ability 

Achievement 



Trait 



20 



14 

trompage 11) 

No. Although all the important words from the title are here, 
there is a better way of listing them. 

We explained in Lesson 2 that pre -coordination is an advantage 
whenever a very general term is used. K you list such pre -coordinations 
as candidate terms, it will make for greater ease and efficiency in trans- 
lation. 

There are four very general terms in this title: Ratings, Scores, 
Prediction and School. 

K is therefore better to list "Likability ratings" and Ability scores" 
than to list the four words separately. In the list that you chose "Prediction" 
and "School" are combined in one phrase with "Achievement." On the whole, 
pre -coordination in ERIC should be limited to two terms, so it is better to 
breakup this phrase into "School achievement" and "Achievement Prediction." 

Please continue on 12. 



15 

(from page 9) 



No. The following definition shows that the description ’’Case 
study” is not applicable to this article. 

’’The case study ... is an approach which views any social unit 
as a whole. Almost always, this means of approach includes the develop- 
ment of that unit which may be a person, a family or other social group . . . ” 

Please return to 9 and choose another answer. 



16 

(from page 13) 

You feel that the term ’’ability” is the genus of ’’likability. ” 

No. Likability is nortjmlly thought of as a quality of a person 
rather than an ability. 

Please go back to 13 and select another answer. 



17 

(from page 9) 



No. The following definition shows that the description "Surveys" 
does not apply to this article: 

"The systematic collection of data from populations or samples of 
populations through the use of personal interviews or other data gathering 
devices." 

Please return to 9 and choose another answer. 



o 



18 

(from page 13) 



You feel that the term ’’achievement” in the first paragraph is 
genus of the species ’’likability. ” 

No. Likability is normally thought of as a quality of a person, 
rather than an achievement. 

Please return to 13 and select another answer. 



19 

(from page 9) 



No. It is true that the first 2 paragraphs refer to previous research, 
but this is a common practice and does not warrant treating the document 
as a review of research in the subject. 

Please go back to 9 and select another answer. 



20 

(from page 13) 



You feel that the word "trait" is the genus of "likability". 

Good. The phrase in the third line, "the trait of likability ... ", 
suggests the generic link. 

In case ERICT fails to show the relation between trait and likability, 
we now record it on our form in the following manner, using parentheses 
to show that it is a doubtful term: 



1 


2 


3 


4 


5 


6 > 


Kind of 
Document 


Educational 
Establishment 
(or part of) 


Person 

Studied 


Attributes 
or Actions 
(of terms 
in any 
column 


Curriculum 


Teaching ? 

and C 

Learning \ 
Methods S 
and Admin- 
istration) 








Likability 
{ Trait) 







The introductory statement "AN EARLIER article ..." means that 
generally speakii^ we should not expect to use the key terms here. For 
example, "grade point average" is a measure of achievement which refers 
to another study so we should not list it as a candidate term merely on the 
strength of this paragraph. 

We should remember, however, that "grade point average" may 
occur in the present article and be prepared to use it when it occurs. Such 
terms may also suggest useful concepts which are i mplicit in 'the present 
document, though not explicitly stated. 

The first paragraph says "An earlier article reported (on tests with) 
. . . high school seniors. "High school seniors" should 

be selected as a candidate term J 

be ignored because it is a keyword 

for "an earlier article" ‘ 

not be selected as a candidate term 

yet, but be kept in mind. - 



(from page 9) 



No. The description headed ’’Purpose and Procedure” near the 
beginning of the article is an indication that it does belong to one of the 
categories. 

Please go back to 9 and select another answer. 







22 

(from page 20) 



No. You should not select "High school seniors" as a candidate 
term, since its use here is no more than a brief reference to what is 
described as "an earlier article." But the present document is on roughly 
the same subject and you will do well to bear "High school seniors" in 
mind as a possible or even likely keyword. Attention of this kind can save 
time and effort in indexing. 

Now turn to 26 and go on with the lesson. 



23 

(from page 28) 



You say you can find 6 to 10 terms that reasonably fit into the form 
of the question. 

No. If you included terms like "purpose," "present article, " "report, 
"findings, " "investigation, " etc. , none of these fit reasonably in a question. 
Ignoring the words mentioned above, then, go back to page 28 and try again. 



24 

(from page 20) 



No. Although you should not include the term "High school seniors 
as a keyword for this paragraph (since its only use here is in a brief 
reference to "an earlier article." You will do well to keep "High school 
seniors" in mind as a possible keyword since the present document is 
concerned with a similar topic. Attention of this kind can save time and 
effort in indexing. 

Now go on to 26 and continue with the lesson. 



25 

(from page 28) 



You say you can find 1 to 5 terms that reasonably fit into the form 
of the question. Good, so can we. 

Many terms do not fit reasonably into the framework of a question 
”Have you any documents on x "purpose," "present article," "report," 
"findings," "investigation," (one might say that all educational research 
documents refer to investigations), "which," "used," "actual situation," 
"predictive situation. " "Predictive situation" echoes one you have already 
provisionally accepted (prediction) — but it adds nothing significant to the 
concept implied in this form (predi ction as distinct from Predi ctive ). 

The phrase "The purpose of the present article ..." should alert 
us to the possible presence of an important keyword. We find the heart 
of the matter in the phrase, "an investigation in which formula one was 
used in an actual predictive situation" — but the phrase "formula one" 
cannot be considered a useful keyword. However, it does imply the use 
of "Grade point averages" in relation to "Ability scores" and "Likability 
ratings." We did not use "Grade point averages" as a candidate term 
before, since it referred to an earlier study. May we use it now, in spite 
of the fact that this paragraph does not mention it specifically? 



Yes 


29 


No 


31 




26 

(from pages 20, 22, 24) 

Good. "High school seniors" cannot be justified as a keyword 
for this paragraph, but we may keep it in mind as a possible keyword- 
later. 

Another problem raised by the first two paragraphs is the 
difficulty of using statements of value to define types of documents in 
educational research. Neither quantitative values like . 550 or . 668, 
as found in the second paragraph, nor qualitative values like "high 
correlation" are useful as keywords. To anyone wanting information 
on "high correlation" a document on any degree of correlation may be 
relevant, and a useful search is likely to be for "correlation" regardless 
of value statements. 

Should precise quantitative values be indexed? 



27 

(from page 28) 



You say you have already listed all terms. 

Have you anything on ninth grade ability scores? Go back to 28 



and see if you can find any more that should be considered. 



F 



1 



28 

(from pages 26, 30) 

Your conclusion is that precise quantitative values, such as those 
found in this document, should not be indexed. 

And you’re right. We should not be sidetracked by the mention of a 
particular value for the validation of a certain assumption. The real subject 
is the relation between the phenomena in question (e.g. between likability 
ratings and grade point averages), and any documents discussing this should 

be retrieved. 



PURPOSE AND PROCEDURE 

The purpose of the present article is to report the 
findings of an investigation in which formula one was 
used in an actual predictive situation. 

The ninth-grade students in one of our junior high 
schools served as the subjects in this study. Ability 
scores were available for slightly over 200 of these 
students, since they had been given the California 
Test of Mental Maturity (CTMM) near the end of the 
sixth grade. 



Applying the simple criterion suggested previously (Have you any 
documents on ? ), how many terms can you find in these two para- 

graphs that you haven’t already listed? 

From 6 to 10 terms. 

From 1 to 5 terms. 

All the terms have already 
been listed. 




23 

25 

27 



29 

(from pages 25, 31) 

Correct. In addition to ’’Grade point averages” (which we have now 
decided is relevant to this document), other terms fit into the question: ’’Have 
you anything onjc ?” In the order in which we come across them in para- 
graphs 3 and 4, these terms are: ’’Ninth grade,” ’’Students,” ’’Junior high 
schools,” ’’Ability scores,” ’’California Test of Mental Maturity (CTMM),” 
’’Sixth grade.” You’ll notice that one term, ’’Ability scores” is one you 
already have on your list, so it need not be repeated. 

Each one of these terms fits into the role of representing a class of 
information on which questions may well be asked: e.g . , ’’Have you anything 
on Ninth grade ability scores?” or ’’Have you anything on achievement pre- 
diction in Junior High Schools? ” Now we must ask if each of these terms 
represents new information or information vital to the indexing of this docu- 
ment . 

The first problem is the word ’’students.” This is a very general 
term; the notion of student is already implicit in many of the terms we have 
already accepted (e.g . , high schools, ninth grade). So we will record this 
as a very doubtful candidate term by putting it in parentheses. 

The second problem is the concept of CTMM. This names a specific 
type of test, but the test itself is not described or evaluated in any way in 
this document. Furthermore, it is certain that much fuller information on 
this test is available elsewhere, so for the time being, we can ignore it. 

Which candidate terms can we add to our list? 



Ninth grade. Students, Grade 
point averages. Junior high 
schools. 


32 


Ninth grade. Grade point averages. 


Junior high schools 


34 


Ninth grade. Students, Grade point 
averages. Sixth grade. Junior 
high schools. 


36 



30 

(from page 26) 

No. Neither quantitative nor qualitative values are useful 
as indexing terms in educational research, since they apply to other 
terms much more likely to be highly relevant to a search. That a 
score of . 3 or .8 was made on a test, does not matter in this field 
so much as a description of the test and its use. 

Now go on to 28 and continue with the lesson. 



31 

(from page 25) 



You would be wrong not to use "Grade point averages" as a 
candidate term now, because this paragraph, though it does not refer 
explicitly to "Grade point averages," does imply them in discussing 
"formula one." 

You must always remember to include terms that refer to the 
meaning of the text if those actually mentioned in it are insufficient. 

Now go on to 29 and continue with the lesson. 





32 

(from page 29) 



You say that we can add to our list of candidate terms: "Ninth 
grade”, "Students", "Grade point averages", (’Junior high schools”. 

Yes. You should include "Students", even though it is doubtful . 
You can indicate this by putting it in parentheses. 




You may notice here that "Junior hi^h school" should make 
"School" unnecessary as an isolated term . But we have provisionally 
coordinated "School" with "Achievement" ("School achievement"), so 
at this stage it must be retained. If we find in the translation stage 
that ERICT does not coordinate school (or a synonym for school) with 
achievement, but leaves it on it J own, then "Junior high school" will 
b€' all that is required. 

Please continue on 33. 




33 

(from page 32) 



"Likability" ratings by teachers were obtained on 
the students in the manner described previously, 
except that care was taken to supplement the 
written instructions with a talk to the teachers. At 
this personal appearance, additional explanations of 
the rating technique were given and suggestions made 
as to how to proceed in discriminating among cases 
at first perceived as near-equal in "likability". Stress 
was also put upon the importance of accuracy and inde- 
pendence of judgment in each case. This additional 
effort at orientation and explanation for the teachers 
was done in an attempt to improve the reliability of 
their ratings. 

"Reliability" (of ratings) could be an important concept. However, 
this paragraph merely states that something was done to improve reliability. 
It contains no information that justifies indexing the term. 

Should we include "teachers" as a further candidate term for this 
document? 



Yes, because it is the agent 
of the operation "rating. " 



35 



Yes, because unless it is used, 
the document might be mistakenly 
retrieved in a request for student 
or parent ratings. 



37 



Yes, for both the above reasons. 



41 



No, because we take it for granted 
(in the context of school achievement) 
that the teachers will be doing the 
rating. 



39 



ERIC 



34 

(from page 29) 

You say that we can add to our list of candidate terms: 

Ninth grade, Grade point averages, Junior High schools 
But you We forgotten one candidate term that we really should include in 

our list until we get to the translation stage, even though we consider it 
a doubtful one. 

Return to 29 and select the correct list of candidate terms. 



35 

(from page 33) 



You say that we should include ’’Teachers” as a further candidate 
term because it is the agent of the operation ’’rating.” 

You’re partially right, but there is a better answer. Please return 
to 33 and select it. 



36 

(from page 29) 

You say we are left with the following to add to our list of 
candidate terms: 

Ninth grade, Students, Grade point averages. 

Sixth grade. Junior high schools 

But we*ve decided to ignore ”Sixth grade” because it simply describes 
the age at which the 9th grade students received their ability rating. 

It seems quite insignificant as a subject of this document. 

Please return to 29 and select the correct list of candidate terms. 



37 

(from page 33) 



You say that we should include ’’Teachers” as a further candidate 
term because, unless it is used, this document might be mistakenly 
retrieved in a request for ratings by students or parents. 

You are partially right, but there is a better answer. Please 
return to 33 and select it. 




38 

(from page 41) 



You say we should put ’’Teachers” in Column 3 under ’’ Persons 
studied”. 



No. If you did, it would look like this: 



> 3 


4 


5 


6 


7 1 


1 8 


(f Person 
S Studied 








Testing, 

Evaluation, 




b Teachers- 





















If you record it in the way shown above, the index term would mean 
the rating of teachers. To avoid confusion, we must list ’’teachers” in 
column 8, as an agent, like this: 



} 3 


4 


5^ 


6 


7 


8 










Testing, 

Evaluation, 

Measurement 


Agents or 
Means (of 
Col. 7) 










Rating 


-Teachers 



This indicates that the teacher is doing the rating, as he is the 
agent. Now you have your ninth candidate term. Please continue on 43. 



39 

(from page 33) 



You say that we shouldn't include " Teachers " as a further candidate 
term for this document because we take it for granted, in the context of 
school achievement, that the teachers will be doing the rating. 

But this sort of rating might be done by students, parents or other 
assessors. 

Please return to 33 and select another answer. 



40 

(from page 44) 



No. These two paragraphs are concerned with minor details of 
test procedure. There are no new terms here for indexing. 



Please continue on 46. 



41 

(from page 33) 

You say we can include "Teachers" as a candidate term, not only 
because it is the agent of the operation "rating," but also because, unless 
it is indexed by this term, the document might be mistakenly retrieved in 
a request for ratings by students or parents. 

Right. Paragraph 5 refers to "Ratings by teachers." We should 
not take it for granted that teachers did the rating. It could be done by 
students, parents or other assessors. 

If "Teachers" is not indexed, the document would be mistakenly 
retrieved in a search for ratii^s by students or parents, and it would not 
be retrieved in a search which specified rating by teachers. 

In order to make clear that the subject we have to index is "Rating 
by Teachers," and not "Rating of Teachers," in which column should we 
put "Teachers"? 

38 



In column 3 (Person studied) 

In column 8 (Agents or Means) 



43 



42 

(from page 47) 



You say you would consider "Biserial correlation" as a candidate 
term for this document. 

We can’t agree with you. The discussion on tests and statistical 
techniques are applicable here. Biserial correlation is an example of 
an analytical technique which was applied to the predictive results; although 
the document e^qplains how it was done, it is not an exhaustive or general 
description. Moreover, we are referred to a book on psychological 
statistics in which this method is probably described at some length. 

Now go on to 45 and continue with the lesson. 



43 

(from pages 41, 38) 

You say we should put "Teachers” in Column 8 under Agents. 

Yes. By indicating that the teacher is the agent, we read this as 
rating by teachers. 

To avoid confusion, then, we list our ninth candidate term as follows: 




Paragraph 5 also gives details of how the investigators explained 
the method of rating, but it does not give sufficient information on the concepts 
jmentioned (written instru cti ons, rating te chniques, reliability, etc. ) to justify 
indexing those terms. 

When we discussed exhaustivity in Lesson 2 we stated that the choice 
of terms is not settled simply by recognizing whether a term will function 
well as the name of a class, but also by deciding how many of these classes 
we can afford to recognize and still index efficiently and economically. We 
also stated that there is no easy solution. In general, you should index all 
those concepts to which the document refers prominently, but avoid indexing 
those to which only passirig reference is made. When in doubt, ask yourself. 
"If I were asking for information on x , would I like to see this document, 
or would I regard the information it gi\ 9S on x not worth the retrieval of 

the document?" 



Continue on 44* 



The above part of the experiment was carried out in 
early December. During the latter part of the same 
month, the ratings were evaluated numerically, both 
ability scores and ”likability" ratings converted to 
standard scores, and the calculations of formula one 
made to obtain the predicted scores of the students 
at the end of the first semester, which was to occur 
in early February. Solely on the basis of these 
predicted grade-point averages, the students were 
listed in rank order, beginning with the student with 
highest predicted score. On January sixth, a copy of 
this list was left with the principal to retain as an 
indication that the experiment was ’’predictive” in 
nature. 

Near tlie middle of February, and after report cards 
had been returned, the grade-point averages of the 
students (derived by letting A = 5, B=4, C-3, 

D = 2 and F = 1) were calculated and the students 
ranked on this basis, beginning with the student of 
hio*hest average. Thus, two lists of the students - 
the one of ’’predicted” and the one of ’’actual” rank 
order of achievement — were available for comparison. 

Do paragraphs 6 and 7 contain any new terms for indexing? 



No 



46 



45 

(from pages 47, 42) 

You say you would not consider ’’Biserial correlation” as a candidate 
term for this document. Neither would we. ’’Biserial correlation ’ is an 
example of an analytical technique which was applied to the predictive re- 
sults; although the document explains how it was done in the particular 
circumstances, it is not an exhaustive or general description. Moreover, 
a reference is made to a book on psychological statistics in which, pre- 
sumably, this method is described at some length. 

Paragraphs 11 through 14 are headed by the term ’’Discussion, 
and though they would not be expected to provide new candidate terms they 
may clarify the relations between those you have already recognized and 
their significance. For example, Paragraph 11 refers to ’’ability tes t 
scores." 




Paragraph 11 also refers to the predictive power of ’’Equation 
one.” (referred to in Paragraphs 3 and 4 as ’’Formula one”) 

We should 

include ’’Equation one” as a candidate 
term. 

include ’’Formula one” as a candidate 
term 



not include a candidate term for 
this at all. . 



48 



46 

(from pages 44, 40) 

Good. These two paragraphs are concerned with minor details 
of test procedure which do not require indexing. 



RESULTS 

First, a word about the reliability of the teacher 
ratings in this e3q>eriment. Using the split-half 
method, wherein the average of the first half of 
the ratings for any subject is paired with the 
average of the second half, the obtained correlations 
between the halves was .751. When corrected by the 
^ear man -Brown formula, the coefficient of reliability 
becomes .857 which, for ratings of this nature, is judged 
to be quite satisfactory. 

Paragraph 8 introduces a section headed "Results." An index 
description must include the information content of the whole document, 
not only the results of an e3q>eriment. In an extreme case we should 
not refuse to index the account of a useful investigation because its re- 
sults were inconclusive or its hypotheses proved incorrect. 

Paragraph 8 refers again to the reliability of ratings. The "split 
half method" and the "Spearman Brown formula" were used, but no other 
information is given. 

The general principles governing our choice in such matters have 
already been explained. Mere reference to the use of a particular test 
or technique in an investigation does not usually justify its indexing. Most 
general accounts of reliability assessing would give substantially more 
information than this, and the information given here does not seem sufficient 
to justify its recognition. 

Now go on to 47. 



47 

(from page 46) 



The method of biserial correlation (3) was used to 
analyze the results on prediction of achievement. 

To do this, the list of students based on actual rank 
order of achievement was dichotomized by choosing 
the grade-point average of 3. 00 as an arbitrary di- 
vision point. This gave an ’’Upper” group made up 
of those with grade point average above 3. 00 and a 
’’Lower” group composed of those with grade point 
average 3. 00 and below. The biserial _r between the 
dichotomized variable of actual achievement and the 
predicted achievement scores was .783. The ratio 
of the variance of ’’predicted” to ’’actual” achievement, 
then, becomes . 613. This means that under the con- 
ditions of this study, use of the variables of ’’likability” 
ratings and ability scores in combined form has made 
it possible to account for about 61 percent of the 
achievement variance. 

The biserial technique of analysis used in this case 
not only gives an approximate degree of relationship 
betwe en t he var^iables involved but also affords a con- 
venient and instructive way of perceiving the results. 
Figure 1, for example, shows the distribution of the 
’’Lower” and ’’Upper” groups of students along the 
y — or predicted score — axis. The means of the com- 
bined, ’’Upper” and ’’Lower” groups (My, and M^, re- 
spectively) and the indicated frequencies within each 
interval show quite clearly the degree of overlap and, 
thus, the size of error in prediction. 



Would you consider ’’Biserial correlation” as a candidate term 
for this document? 



Yes 



42 



No 



45 






1 



48 

(from pages 45, 51, 53) 

Good. The idea of ’’Formula” or ’’Equation” is far too general 
for indexing, and the detail given in this document does not justify its 
inclusion. In any case, we have already listed the concept to which the 
formula applied: ’’Achievement prediction. ” 



Equation one was developed usii^ data obtained on 
high-school seniors and indicated at that time ”in a 
retrospective sense” the combination of ability test 
scores and ’’likability” ratings that would have given 
the best prediction of grade-point averages for these 
students. The fact that use of this equation has re- 
sulted in reasonably good prediction for the achieve- 
ment of ninth-grade students offers some, but not 
conclusive, evidence for its general applicability at 
the high- school level. There should be similar ex- 
periments with it at other high- school levels, including 
the seniors, before definite conclusions can be drawn. 
However, the close similarity of results obtained be- 
tween ’’actual” and ’’retrospective” uses of equation one 
in the two experiments conducted so far tends to strength- 
en one’s belief that the method can be used with some 
degree of success at more than one high- school grade 
level. For instance, the percentage of ’’errors in pre- 
diction” among the highest 100 students in the two cases 
is essentially the same (21 and 19). 



This paragraph suggests that ’’Equation One” may be applicable 
to high school grades in general , not just to seniors and 9th grade. 

Does this sentence indicate a need for a change in our candidate 
term ’’Junior high schools” ? 



Yes. ’’Junior high schools” sho 
be broadened to ’’High Schools”. 



Junior high schools” should 



52 



No. This sentence does not seem 
to justify the change. 



50 





49 

(from page 50) 



No. You should not include "Errors" as a candidate term. It is 
the antonym of "Reliability" and at the very least would have been rep- 
resented by that term. But since "Reliability" has already been rejected 
there is no point now in including either term. 



Now go on to 55. 



50 

(from pages 48, 52) 

Yes. li the sentence gave significant information, we should 
have to broaden our candidate term "Junior high schools" to "High 
schools," but it doesn't. For anyone who wanted to be sure of seeing 
everything that had some relevance to achievement prediction in high 
schools, the search program could include Junior high schools as well 
as High schools. 

A well established indexing rule is that opposites should be 
treated as synonyms i.e. they should be represented by the same index 
term. For instance, "tolerance" and "intolerance" may both be indexed 
by the term "tolerance. " Of course, a reference is made from the reject- 
ed form to the one accepted (Intolerance use TOLERANCE). 

In the last sentence of Paragraph 12 the term "Errors" is an 
antonym of "Reliability" — a concept that we considered and rejected 
on the score of inadequate information given. 

Should we use "Errors" as a candidate term? 

Yes 49 

No 55 



o 



51 

(from page 45) 



Although the document does give the equation and does not just say 
that it was used, and although it refers by name to both ’’Formula One” 
and ’’Equation One,” the equation is not described sufficiently, or even 
named clearly. There is no evaluation or new information. It is therefore 
no use as a candidate term. 

Now turn to 48 and go on with the lesson. 



52 

(from page 48) 



You feel that this indicates the need for a change in our candidate 
term "Junior high schools" which should be broadened to "High schools. " 

No. K this amounted to significant information we should have to 
broaden our candidate term "Junior high schools" to "High schools, "but 
this sentence does not seem to justify the change. For anyone who wanted 
to be sure of seeing everything that had some relevance to achievement 
prediction in high schools, the search program could include Junior high 



schools as well as High schools. 

Now you may continue on 50. 



53 

(from page 45) 



Althoiigh the document does give the equation and does not just say 
that it was used, and although it refers by name to both ’’Formula One” and 
’’Equation One,” the equation is not described sufficiently, or even named 
clearly. There is no evaluation or new information. It is therefore no use 

as a candidate term. 

Now turn to 48 and go on with the lesson. 



54 

(from page 56) 

You say that ’’Attitude’' and ’’Achievement” are the two new words 
we can add to our list of candidate terms. 

Check your list ~ you should have one of these already listed. 
Turn back to 55 and try again. 



55 

(from pages 50, 49) 



Good. "Errors" would in any case have been represented by its 
antonym "Reliability," and we have already decided to reject this term. 

In spite of the favorable results obtained so far with 
equation one there is still almost 40 percent of achieve* 
ment variance to be explained. This means that other 
variables related to achievement must be found and the 
manner of their contribution studied before increased 
accuracy of prediction can be assured. A step in the 
search for other potentially useful variables was made 
in the present study. In the case of 11 students, for 
whom predictions based on equation one were most in 
error, the teachers were asked to supply additional 
information. For those falling below predictions, the 
teachers were asked: "Why did these students not 
make better grades?" For those doing better than 
predictions, the teachers were asked to "Describe 
these students in terms of their attitude toward school 
work. " Some of the replies are mentioned here. 

(a) Why those students falling below predictions 

did not make better grades: 

"Little class recitation." 

"Occasionally doesn’t do homework. " 

"Lacks interest, skipped school. " 

"Slow at getting make-up work in. " 

"Is satisfied with less than perfection. " 

(b) Attitude of those students who exceeded predictions: 

"Very studious, always makes up her work." 

"Very diligent worker." 

"Works hard for what she gets. " 

"Works well in class. " 

"Very conscientious. " 

On a purely qualitative basis, these inversely related 
statements seem to have most in common the factor 
of "achievement motivation" which in the present context, 
would, appear to have the positive meaning of "desire 
to make good grades." By somewhat different means, 
others have arrived at similar variables (1, 2). 

This refers to the problem of those students who did not "achieve," 
according to the ejqpectations of the predicting teachers. 



Please continue on 56. 



56 

(from page 55) 

The term ’’Achievement variance” is too general to foe useful. But 
we should look for a term to refer to the small test now described. The 
description of the procedure gives no keywords until the phrase ’’Attitude 

toward school work” occurs in the next to last sentence before the replies 

> 

are listed. This was the basis of the question put to students who exceeded 
the predictions; no good indexing term offers itself to define the question 
put to those who fell below prediction. 

But in paragraph 14 the common factor between students exceeding 
and failing prediction is described as ’’Achievement motivation, ” and this 
phrase is a good example of what the indexer looks for — a term which 
succintly conveys an important concept . Moreover, a reference to two 
other articles at this point discloses a further use of the term ’’Motivation. ” 
Again, the problem of whether to link or not arises, since the motivation 
refers to achievement and not, say, to likability. 

Which two new terms can we add to our list? (they may, of course, 
be linked to terms we have used already) 

Attitude and Motivation 58 

Attitude and Achievement 54 



Achievement and Motivation 



61 



4 ! 



wSBmSm 



SB 



57 

(from page 60) 

No. You have used the plural form ’’Grade-point averages” instead 
of the singular form that ERICT prefers. You should always check care- 
fully with ERICT to make sure that you have the correct form and spelling 
of the term. 

No turn to 64 and go on with the lesson. 



58 

(from page 56) 



\ 



You say that "Attitudes" and "Motivation" are the two nev; words 
we should add to our list of candidate terms. 

Right. Since they are both very general terms they should be linked 
with the terms to which they refer. 



5 3 
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That concludes the text of the document, but you will notice in your 
copy that the document includes a diagram entitled "Predicted Score Axis. " 
When a document includes tables, diagrams, etc. , these also should be 
checked for keywords. Usually they duplicate information already referred 
to in the text, but they may provide more succint terms . In this diagram, for 
example, we find "Grade-point average" used as a measure of achievement. 
This confirms our previous recognition of the term. 

You should now have a complete list of all the candidate terms we 
have discussed. Check your list, then look on 59 and see how it compares 
with ours. 



Continue on page 59. 



59 

(from page 58) 



Here is our list of candidate terms for this document* Is yours 
exactly the same? If not, please correct it. 




We can now look up each of the candidate terms in the ERIC 
Thesaurus. As we discuss each term, be sure to list the translation on a 
separate piece of paper. The final list will contain the indexing terms from 
ERICT and any new terms that you are recommending.* Remember to make 
a note beside these terms. . 



Turn to 60 and go on with the lesson. 



60 

(from page 59) 



Translation of Candidate Terms into SIvIO Language 

We have 11 terms to translate into the language of ERICT represent- 
ing different kinds of problems in translation. We shall not deal with them 
in their present order, but in the order of the kinds of problem, beginning 
with the easiest. 

The first term in our list, "Experiments, " We shall not consider 
further since it is not a subject word. 

There are some terms which are simple to translate. If we look 
for "Junior high school" in ERICT, we find the entry: 

JUNIOR HIGH SCHOOLS 
RT Senior schools (etc. ) 

Our choice of candidate term is thus confirmed by ERICT, and we use the 
term (in the plural form approved by ERICT) as the correct indexing term. 

We have a candidate term "Grade-Point average. " Look in ERICT 
and choose an indexing term for this 

Grade-point averages 57 

Grades (scholastic) 62 

Grade-point average 



64 



61 

(from page 56) 



You say that ’’Achievement” and ’’Motivation” are the two new words 
we should add to our list of candidate terms. 

Check your list — one of these should already be listed. Return to 
55 and try again. 



62 

(from page 60) 

No. You have chosen a term that ERICT lists, but you failed to 
notice that GRADES (SCHOLASTIC) gives GRADE-POINT AVERAGE as an 
NT, and this is the very term that you want. Always check the references 
at a term you consult to see if a better term is given there. ^ 

Now turn to 64 and go on with the lesson. 



63 

(from page 64) 



No. You have chosen to use the candidate term as it stands, but if 
you had looked in ERICT carefully you would have seen the entry 

Achievement Motivation 
USE MOTIVATION 

Always look at references carefully to make sure that the term you 
have consulted is available and appropriate. 

Now go on to 67 and continue with the lesson. 



64 

(from pages 60, 57, 62) 



Sometimes the candidate term is listed in ERICT, but only as an 
unused term, with a reference to the preferred term. For instance, our 
candidate term "School achievement" appears as: 

School achievement 

Use Academic achievement 

When we look up Academic achievement we find: 

ACADEMIC ACHIEVEMENT 
UF School achievement (etc. ) 

So ACADEMIC ACHIEVEMENT is our translation of the candidate 
term "School achievement. " 

What is the correct translation of "Achievement motivation"? 



Achievement motivation 


63 


Motivation 


67 


Achievement and Motivation 


68 



65 

(from page 67) 



No, ’’Ninth grade” does not appear in ERICT and you may not use 
it as an index term. If you had looked up”Grade 9” you would have seen 
that ERICT prefers to list all the grades by number under the word GRADE. 
Phrases of two or more words should always be checked under each word 
in case ar inverted form is used, as GRADE 9 is used here. 



Now turn to 70 and go on with the lesson. 



66 

(from pages 70, 72, 75) 



Yes. Several factors point to the recommendation of a new compound 
term for inclusion in ERICT. The only translation already available in the 
thesaurus is ABILITY + SCORING. But variant word forms are normally 
listed, which means that at least we should recommend the addition of "Scores. 
However , there are two reasons for preferring a compound term. "Scores" 
is the kind of general term that is best used in a compound (hence our listing 
it in this way among ^he candidate terms). Moreover, ERICT does give us 
some guidance in that it lists the analogous forms ABILITY GROUPING and 
ABILITY IDENTIFICATION. Since the compound "Ability scores" implies 
tests, we do not need this term which we listed in parentheses in our concept 
indexing. 

Some compound candidate terms seem to be listed in ERICT, but on 
examination they prove to have a different meaning. One of the uses of the 
scope notes and lists of references under each term in ERICT is to show 
you the scope and meaning of each term. 

At the same time it may be that ERICT does not provide you with an 
alternative term to represent your concept accurately, and 3rou may then have 
to suggest a new term for inclusion in the thesaurus. 

For example, our candidate term "Teacher ratings" means "ratings 
by teachers," but the entry in ERICT, TEACHER RATING, clearly confirmed 
by the entry TEACHER EVALUATION to which it refers, means "ratings of 
teachers. " For our meaning the only solution seems to be the use of the 
separate terms TEACHERS and "Ratings. " 

"Rating" does not appear alone in the Thesaurus, though EVALUATION 
does. Since TEACHER RATING is an NT of TEACHER EVALUATION, we 
should use "Rating" as an NT of EVALUATION and suggest it for inclusion 
in ERICT. The translation ol our candidate term "Teacher ratings" should 
therefore be TEACHERS and "Rating" (Suggested as a new term for ERICT). 

What is the translation of our candidate term "Achievement prediction"’ 



Achievement prediction 
Achievement tests 



71 

73 



67 

(from pages 64, 63) 

Good. Sometimes the candidate term will not be listed in the form 
you have in mind, but you should always check first to see if it appears as 
a synonym or in a different form. 

What is the correct translation of ’’Ninth grade”? 



Ninth grade 


65 


Grade 9 


70 


Grades (scholastic) 


69 



68 

(from page 64) 



No. You have not read the references and notes in ERICT at the 
terms you have chosen. Do so, and you will find yourself directed to 
a better answer. Then go back to 64 and try the question again. 



o 

ERIC 

hiaifaifftiiTiTiaa 



69 

(from page 67) 



No, ^’Grades (Scholastic)" has a different meaning. It is true that 
"Ninth grade" is not listed in ERICT, and you were quite right to think 
of looking under "Grade" — but if you had looked a little more carefully, 
you would have seen all the grades listed by numbers just before the entry 
GRADES (SCHOLASTIC). So that you could have used the entry GRADE 9. 

Now turn to 70 and go on with the lesson. 



t 



70 

(fromp ^es 67, 65, 69) 

Good. ERICT does not list ’’Ninth grade” as an indexing term, 
but as with all phrases of two or more words, it is essential to look 
under M the words in the phrase in case an inverted form is used. Here 
the inverted form GRADE 9 is listed in ERICT. 

Although many pre- coordinated terms are listed in ERICT there may 
be compound candidate terms not listed. In each case you must decide 
whether the individual terms are satisfactory, or whether you should 
recommend the addition of a new compound. Remember that we discussed 
the reasons for pre -coordination in Lesson 2. 

What is the best translation of ’’Ability scores”? 



Ability scores 


66 


Ability and Scores 


72 


Ability and Scoring 


75 



o 



71 

(from pages 66, 73) 



Yes. ’’Achievement prediction” refers in ERIC T to a preferred 
term ACHIEVEMENT TESTS, but the two terms have quite distinct 
meanings. It is time to re-examine this entry and submit ACHIEVEMENT 
PREDICTION as a new term. 

Sometimes our candidate term is obviously too long or complex for 
ERICT to include it as a pre-coordinated term. ’’Student attitudes to school- 
work” is a good example. What we must do is to ^lit it up and include 
elements that \ ill cover the same meaning. 

If we look up ATTII JDES in ERICT we find a long and mixed 
batch of NT’s of which School Attitudes and Student Attitudes seem the most 
likely. We can reject School Attitudes as being too general, k* we accept 
Student Attitudes and think of it as a part of the whole description, ’’Student 
attitudes to school work” what shall we use to complete the description? 



School work (new term for inclusion 
in ERICT) and Attitudes. 



74 



Schoolwork attitudes (new term for 
inclusion in ERICT). 



76 



Study and Attitudes. 



78 



72 

(from page 70) 



No. It is true that ERICT usually includes variant word forms 
(which would suggest adding SCORES as well as SCORING), but other 
factors suggest recommending a new compound term. 

Please turn to 66 for explanation. 



73 

(from page 66) 

No. It is true that ERICT refers you from ’’Achievement prediction” 
to ACHIEVEMENT TESTS, but the two terms have quite distinct meanings. 

If one were to retrieve all documents on achievement testii^ every time a 
search was made for achievement prediction, the precision ratio would be 
extremely lov/. If is time therefore, to re-examine ERICT ’S entry ACHIEVE- 
MENT TESTS and its reference from Achievement prediction, and submit 
ACHIEVEMENT PREDICTION as a new index term. 

You should always be ready to consider the submission of new terms 

* • , 

for inclusion in ERICT; its development depends on your jpecialized sub- 
ject knowledge and eiqploration of the literature. 

Now go to 71. 






o 



74 

(from page 71) 



No. You are quite right to suggest the inclusion of a new term for 
this concept, but ’’the list of NT’s under ATTITUDES suggests that a com- 
pound term should be preferred. ” 

On the analogy of ’’School achievement prediction, ” which we split 
up into ’’School achievement” and ’’Achievement prediction” we should split 
’’Student schoolwork attitudes” into ’’Student Attitudes” and ’’Schoolwork 
Attitudes. ” ’’Schoolwork attitudes” is not included in ERICT and we must 
therefore recommend its inclusion on the basis of its presence in the docu- 
ment we are indexing. 

Now turn to 76 and go on with the lesson. 



75 

(from page 70) 



No. It is true that this is the nearest translation you can find 
a lready in the thesaurus, but several factors point to the recommendation 
of a new compound term. 

Please turn to 66 for an explanation. 



76 

(from pages 71, 74) 

Good. On the analogy of "School achievement prediction," (which 
we split into "School achievement" and "Achievement prediction") "Student 
schoolwork attitudes" should be split into "Student attitudes" and "Schoolwork 
attitudes. " "Schoolwork attitudes" does not appear in ERICT and we must 
therefore recommend it for inclusion. 

One of our first candidate terms was "Likability rating." This is a 
central theme of the document, but it does not appear in ERICT, nor is there 
anything like it. We provisionally recorded the generic term "Trait" with 
"Likability," but there is nothing in ERICT under thi^^ term either. 

But "Likability" and "Trait" are characteristics or qualities of 
persons, and if we consult ERICT for characteristics, we find first: 

Characteristics (individual) 

USE INDIVIDUAL CHARACTERISTICS 

and then: 

INDIVIDUAL CHARACTERISTICS 
UF Personality traits 

This confirms our original train of thotight. But is the term INDIVIDUAL 
CHARACTERISTICS too general to use here? We have said already that in 
principle we should try to index specifically and not be content with terms 
which are more general than the concepts we are trying to index. 

You should be prepared on occasion to develop new index terms 
yourself, as we have seen already, but you should do this with caution, 
inspecting either terms for guidance. Let us look for any other specific 
traits already recognized in ERICT to justify recognizing "Likability" an 
index term. 

INDIVIDUAL CHARACTERISTICS 
NT INTEGRITY 

PHYSICAL CHARACTERISTICS 
raises two points that must be considered. 



Please continue on 77. 






r 

77 

(from page 76) 

INTEGRITY is a trait and has been included because a document on 
it has already been indexed; this seems to justify using "Likability” for 
the present document. 

^ The appearance of "PHYSICAL CHARACTERISTICS" reminds us 

^ that there are "levels" of genus/ species relationships, and that NT relations 

are usually shown "one step at a time." If we look under PHYSICAL CHAR- 
ACTERISTICS we find that it, in turn- has its own species: 

PHYSICAL CHARACTL.tlSTICS 
NT AGE 

SEX (etc.) 

We must ask ourselves if there should be an intermediate term between 
INDIVIDUAL CHARACTERISTICS and "Likability," because the indexer in 
sv^gesting new terms for inclusion in ERICT should be familiar with the 
reference structure. But we have already seen that ERICT has rejected 
"Personality traits" (preferring INDIVIDUAL CHARACTERISTICS) and 
since this would have been the most likely intermediate term, we assume 
that "Likability" can be regarded as a direct species of INDIVIDUAL CHAR- 
AC'^ERISTICS. 

V/hat is your final lecision on "Likability"? 



We should reiect it because 


it is not in ERICT. 


79 


We should accept INDIVIDUAL 


CHARACTERISTICS. 


80 


We should accept "Likability" 
and recommend its inclusion 
in ERICT. 


82 



78 

(from page 71) 

No. "Study" is too general a term in ERICT, as can be seen by 
looking at the references listed in that entry. There is no reason why 
you should not recommend a new term for inclusion in ER^JT if the 
concept is not represented. 

Now go back to 71 and try the question again. 




79 

(from page 77) 



You have decided to reject ^Likability’’ as an indexing term because 
it was not listed in ERICT. 

But ERICT is not complete. The decision must be made now ac to 
whether this term is necessarv for retrieval of the document. 

Return to 76 and review the discussion on ”Likability," then see if 
you can make a better decision. 






80 

(from page 77) 

You have decided to accept INDIVIDUAL CHARACTERISTICS 
to describe the concept of ’’Likability” for this document. 

No. You should not be content with terms which are more general 
than the concept you are trying to index. You should select the most 
specific term that the inquirer is likely to use. 

Return to 76 and read the discussion on ’’Likability” again — then 
see if you can make a better decision. 






81 

(from page 82) 



No. We were very doubtful in the first place about including such 
a general word as a candidate term, and the translation process shows 
clearly that it is so general in ERICT as to be virtually useless on its 
own as an indexing word. We shall abandon this term. 

Now turn to 83 and go on with the lesson. 



82 

(from page 77) 

You have decided to accept ’’Likability" as an indexing term and 
recommend its inclusion in ERICT. Gtood. The precision with which the 
ERIC store can be searched is dependent upon the specificity of the indexing. 

We must finally decide whether to recommend "Likability" as a 
separate term (for post coordination with rating) or whether to recommend 
the compound form ’’Likability rating. ” Once again we are dealing with a 
very general term (rating), and with the added guidance of ERICT, which 
includes Achievement Ratb ; *^.nd Teacher Rating, we should choose 
’’Likability Rating. ” 

There is one last candidate term to be considered: ’’Students.” 
There are five and a half columns of NT’s and RT’s under this term, but 
no scope note to tell us when this term should be used. Should we use it 

here? 



No 



83 



83 

(from pages 82, 81) 



Good. Here is the final list of descriptors for the document ’’The 
Use of Likability Ratings and Ability Scores in the Prediction of School 



Achievement.” Compare it with your own 


1. 


JUNIOR HIGH SCHOOLS 


2. 


GRADE-POINT AVERAGE 


3. 


ACADEMIC ACHIEVEMENT 


4. 


MOTIVATION 


5. 


GRADE 9 


6. 


ABILITY SCORES 


7. 


STUDENT ATTITUDES 


8. 


TEACHERS 


9. 


RATING 


10. 


ACHIEVEMENT PREDICTION 


11. 


SCHOOLWORK ATTITUDES 


12. 


LIKABILITY RATING 



You have recommended the following for inclusion in ERICT: 
Ability scores 
Rating 

Achievement prediction 
Schooiwork attitudes 
Likability rating 

Each of these are terms that may be used in the search, so they 
should be included in ERICT. Your recommendations should be noted as 
you go through the translation stage. Your supervisor will describe the 
procedure for completing the appropriate form to be sent to ERIC Central. 



Continue on 84. 



r4 



Major and Minor Terms . There remains the final task of selecting 
5 major terms from the 12 translated terms. We esqplained earlier that the 
division of indexing terms into "major" and "minor" was a form of weighting . 
Only those terms which refer to concepts of major importance in the docu- 
ment should be included as "major terms. 

The best way to find your most important terms is to ivrite down a 
summarization of the document’s main theme . For this document it would be: 
(Use of) Likability Ratings (and) Ability Scores (for) Prediction (of) Academic 
Achievement (as shown by) Grade-Point Average. 

The indexing terms we have used for these ideas are: 

Likability rating:; Ability scores; Achievement prediction; 
Academic achievement; Grade -point average. 

This gives us just five major terms. 

The terms which we excluded from our summarization fall into two 

groups: 

"Student attitudes," "Schoolwork attitudes," "Motivation," "Teachers, 
and "Rating" refer to concepts of minor significance in this document. 

"Junior high school," and "Grade 9" are the "guinea pigs" of the 
e 3 q)eriment. Our concern is not so much with them as with the idea of 
establishing a correlation between the factors of likability, ability, and 
grade-point averages. The suggestion that the correlation may well apply 
outside this narrow range supports the view that the sample is not our main 
concern. 



Continue on 85. 




r 



\ 



If we mark our major terms with one asterisk, the final list of 
indexing terms for this document is: 

Indexing Terms For "The Use Of Likability Ratings And Ability Scores In 
The Prediction of School Achievement. " 

LIKABILITY RATING* 

ABILITY SCORES* 

GRADE-POINT AVERAGE* 

ACHIEVEMENT PREDICTION* 

ACADEMIC ACHIEVEMENT* 

RATING 
GRADE 9 

JUNIOR HIGH SCHOOLS 
TEACHERS 

STUDENT ATTITUDES 
SCHOOLWORK ATTITUDES 
MOTIVATION 

This con^letes our detailed demonstration of indexing an ERIC 
document. You may now go on to Lesson 4. 



% 
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INDEXING FOR ERIC 
Lesson 4 



EXERCISES IN INDEXING 



Indexing Exercise No. 2 



Donaldson, Mary Jane and Harvey, John F. ’’Library School Instructor 
Evaluation.” College and Research Libraries , November, 1966, pp. 470-477. 

Look through the article and select from the answer choices below 
a term that describes the kind of document. 

Controlled e^iperiment 
Research review 
Case study 
Survey 
None of these 

When you have made your choice turn to 2. 



2 



This article does not belong to any of the four major types (Controlled 
experiment, Research review, Case study, or Survey). 

The title is "Library School Instructor Evaluation." Write on the 
form the concepts you decide to index and turn to 3 to check the answer . 



o 

ERIC 



From the title we obtain two concepts: "Library school" and "Instruct- 
or evaluation. " 

But "Library school" describes an establishment or department in 
which library science is taught. Since the subject is the purpose of the 
school, it seems best in such cases to index the subject , rather than the 
educational establishment. If the subject of the article was administrative, 
the term "library school" would be preferable. Here, however, we will use 
"Library Science." Now, if you have not done so already, enter these terms 



on zhe form. 


1 2 


3 4 


5 6 


00 






P er s on 
Studied 


Cur riculuiTL 


Measurement 






Instructor 


Library Sci. 


--Evaluation 





"Instructor" is entered in column 3 as the person being evaluated. 

If the instructor had been evaluating someone else, the term "Instructor" 
would naturally have gone in column 8 as the agent of the evaluation process. 

Now please turn to 4 and read the first paragraph of the document 
to see what concepts it contains. 



Paragraph 1 contains a brief description of the investigation. 



In 1960 the Drexel Institute of Technology graduate 
school of library science initiated a program of eval- 
uation of faculty members by students. Student 
ratings are obtained for each section taught by full- 
time instructors during their first three quarters at 
Drexel, and for the first four courses taught by part- 
time instructors. Normally the evaluations are ob- 
tained in the last class meeting before the final exam- 
ination. 

List on your form the concepts you can find in this paragraph. 

When you have done this, turn to 5 and compare your results with 



5 



Paragraph 1. "Drexel Institute of Technology" should be listed 
as an identifier at the bottom of your form. 

"Faculty" suggests a synonym for "Instructors," which could be 
recorded on the form as a possible aid when you come to the translation 
stage. 



The library school is described as a graduate one, but the term 
properly describes the students rather than the school. We can therefore 
list "Graduate students" as the agent of "Ratings." "Ratings" by Graduate 
students adds further important information. Possible additions are "Part- 
time instriTctors" and "Full-time instructors," but their inclusion is not 
warranted merely on the strength of the first paragraph. 



Now enter these new terms on your form if you have not already. 



1 ^ 


3 


4 


5 


6 


7 


8 


s Educational 
JEstablish- 
) ment 
/(or part of) 


Person 

Studied 


Attributes 
or Actions 
'of terms 
in any 
column) 


Curriculum 


Teaching 

and 

Learning 
Methods 
and Admir 
i strati on 


Testing, 
Evaluation, 
- Measure- 
ment 


Agents or 
Means (of 
Col. 7) 


1 


[nstructc 

(Faculty) 


m m m .a 


Library Sci 




Evaluation 
Ratings 


Graduate 

Students 















Identifiers . Drexel Institute of Technology. 
Please turn to 6 and read paragraph 2. 



6 



Paragraph 2 . In this paragraph you will find the purposes of the 

study. 



The evaluations are intended to serve three purposes. 
Primarily, it is hoped that the ratings will aid the in- 
structor in improving his teaching methods by pointing 
out specific areas needing improvement and by reveal- 
ing to the instructor his students^ reactions to him, 
both positive and negative, since ”only by accident 
will the teaching of a man ignorant of the reaction of 
his class be effective.” Second, the ratings assist 
the administration in judging faculty members^ effect- 
iveness. A third purpose is to give students a voice 
in school administration. 



List on your form the concepts that represent the purposes of this 
study, then turn to 7 and see how your decisions compare with ours. 






Paragraph 2. The purposes of the study may be represented by 
the following concepts: ’’Improvement of teaching methods,” ’’Student 
reactions, ” ’’Student participation (in) School administration. ” 



Enter these terms on your form if you have not already done so. 



1 


2 


CO 


4 


5 


6 


7 


rl 





Improvement 


methods 






Student- - Reactions 








Student- -Participation 






School-- 









Please turn to 8 and read paragraph 3. 



8 



Paragraph 3 , This discusses the reliability and validity of student 
ratings. 



Since the 1920’ s, and especially in the last two decades, 
student evaluation of instructors has received increas- 
ing attention from educators and researchers. In 
general, this research has shown student evaluation 
to be reliable and valid. In his survey of the subject 
in the ’’Handbook of Research on Teaching,” H.H. 
Remmers cites various studies which have shown 
that student ratings were not appreciably influenced 
by the difficulty of the course, by the halo effect, by 
the grades given to raters, or by the instructor’s pop- 
ularity in extracurricular activities. Remmers cites 
further studies showing when twenty-five more ratings 
were averaged they were as reliable as the better 
mental and education tests, and when alumni graduated 
ten years earlier were asked to rate their college in- 
structors, their ratings agreed substantially with those 
of the same instructors by students currently enrolled. 



List on your form any new concepts you can find in this paragraph 
and then turn to 9. 



41 






Paragraph 3 refers to earlier studies and provides no indexing 
terms for this article . Continue with Paragraph 4 and examine it for 
concepts. 



Paragraph 4 . 



The use of student evaluation is apparently widespread: 
in 1960 Stecklein cited evidence that 320 colleges and 
universities in the United States had used student ratings. 
Despite the amount of attention given student ratings on 
the undergraduate level, however, this subject has re- 
ceived little attention at the graduate level and apparently 
none in library science. Yet it would seem that ratings 
might play a particularly important role in library edu- 
cation since, like instructors in other professional 
schools, most library school instructors have been 
trained primarily as professional practitioners rather 
than as teachers, and sometimes ’’the teachers drawn 
from the ranks of the profession are inadequately pre- 
pared for teaching. ' For the instructor with little ex- 
perience or training in teaching methods, student ratings 
can point out areas on which he must concentrate to in- 
crease his effectiveness. 



When you have done this turn to 10. 



Paragraph 4 like Paragraph 3, refers to earlier studies and pro- 
vides no indexing terms for this article. 

Paragraph 5 . 

With the hope that a study of the results of student 
evaluations might provide insights into the problems 
of teaching library science on the graduate level, 
a study was made of the five-year Drexel evaluation 
program. The data upon which the study was based 
were limited in several aspects: in the first place, 
since ratings were generally obtained only for part- 
time and full-time instructors while they were still 
relatively new to the job, they do not necessarily 
provide a representative sample of all sections 
taught; second, the form of rating sheet has under- 
gone slight m.odifications; and third, in a few cases 
not all of the data was recorded on the master sheets. 



The concept here has to do with the kind of document, even though 
it does not describe the whole document , so it may be conveniently listed 
in the first column. 

Compare your reaction to this with ours on 11. 



11 



Paragraph 5 presents a new concept — ’’data. ” This is a very 
general term and examination of the tables in the article shows that we 
could use the more precise term, ’’Statistical data. ” This is a form-of- 
presentation term, similar to our major categories of Controlled experi- 
ment, etc. Although it does not describe the whole document, it may be 
conveniently listed in column 1 (kind of document) as follows: 



1 2 


3 


4 


5 


6 




7 


8 




Kind of 
Document 



















Statistical 

Data 



We are continuing to note these form words during concept indexing 
so that you learn to distinguish them from subject words. At the translation 
stage we shall ignore them, since ERICT contains only subject words. 

Turn to 12 for paragraphs 6 and 7. 






4 - 



o 

ERIC 



12 



Paragraphs 6 and 7 . 



The evaluation forms filled out anonymously by the 
students consisted of a list of attributes generally 
recognized to be associated with effective teaching. 
For each attribute students were asked to rate the 
instructor on a four-point scale — excellent, good, 
fair, and poor. The rating unit was the individual 
class section, so if an instructor taught three class 
sections in a given quarter he received three separate 
ratings. In compiling the scores for each section 
a master sheet was made which recorded the number 
of times each point on the grading scale v/as marked 
for each attribute. The total number of marks for 
each point on the grade scale was then figured. The 
final rating score was the percentage of all the marks 
which were at the excellent and good points of the 
scale. For instance, if thirty students in a section 
made a total of 200 evaluation decisions, 180 of which 
were at the excellent and good points of the scale, the 
instructor's rating for that section was 90 per cent. 



Also recorded on the master sheet was a list of all 
voluntary student comments and the rank of that 
section compared with all other sections rated that 
quarter. At the end of the quarter - after all course 
grades had been turned in - a report was given to the 
instructor on an evaluation form, and a copy was kept 
by the administration. 



These paragraphs are mainly concerned with the details of the 
rating procedure. There is only one new concept here of value for indexing. 
When you have decided on your choice turn to 13. 



13 



Paragraphs 6 and 7. The only new concept here of value for indexing 
is ^’Effective teaching/^ which we can list as follows: 



1 


7 

IM 


3 4 5 


6 7 


8 








Attributes 
or Actions 
(of terms 
in any 
column) 


Teaching 

and 

Learning 
Methods 
and Admin- 
istration 







Effective Teaching 



Paragraphs 8 through 11 . These paragraphs add nothing new, so we 
can continue with Paragraph 12. Please turn to 14. 



Here is Paragraph 12: 



3. Was there any connection between favorability of 
ratings and time of year ? In the Colorado State College 
of Education survey summer school courses v/ere rated 
higher than those taught during the regular college year. 
The same trend was evident at Drexel where summer 
quarter course ratings averaged 82.5 per cent while fail, 
winter, and spring quarter averages were 79 per cent, 
78. 8 per cent, and 78. 3 per cent respectively. More 
striking than a comparison of average scores, however, 
was the fact that while summer quarter ratings account- 
ed for only 28 per cent of all the ratings they accounted 
for 68 per cent of the twenty-five highest ratings. 



This paragraph compares summer schools with regular offerings. 

The only difficulty here is that although ’’summer school” is a recognized 
term, there is no special way of referring to regular courses. 

What, then, would you list as candidate term(s) from this paragraph? 
Compare yours with ours on 15. 



15 



Paragraph 12. We mentioned that although "summer school" is a 
recognized term, there is no special way of referring to regular courses. 
This is one of many cases in which the regular, or normal, state is 
usually taken for granted and is mentioned ejcplicitly only when a 
comparison is made. There is a comparison here, but we must remen- 
ber that indexing terms should represent those concepts likely to be in 
eluded in inquiries. It is doubtful that "regular courses" is such a con- 
cept. Furthermore, any discussion of summer schools would at least 
imp ly comparison with regular offerings. We conclude that it is sufficient 
to index only the concept "Summer schools." 



1 



2 .3 4 5 1 

Teaching 

and 

Learning 



Methods 
and Admin- 
istration 



Summer 

Schools 



Now turn to 16 and read Paragraph 13. 



16 



Paragraph 13 deals with the comparison of men and women 
instructors. What would you use as candidate term(s)? 



4. Which groups of instructors scores highest? A 
comparison was made of the scores received by women 
(48 per cent of all ratings) and by men (52 per cent). 
Table 3 shows the mean rating for women (81.7 per 
cent) to have been slightly but not significantly higher 
than that for men (79.2 per cent). Male instructors 
made up half of the top twenty-five instructors but 
two-thirds of the bottom twenty-five instructors. 



After you have made your decision, please turn to 17. 



o 



Paragraph 13 deals with the comparison of men and women instruct 
ors. We can record this as follows, with a note that the inclusive term 
”Sex'' may be used in the Thesaurus. 



1 2 3 4 5 6 1 8 

Testing, 

Person Evaluation, 

Studied Measurement _ 



Men Instructors) 
Women Instructors) 
(Sex) 



Comparison 



Please turn to 18 for a discussion of the next paragraph. 



a 



1 



18 



Paragraph 14 . Keeping in mind that full-time instructors and 
part-time instructors bear the same relation tc each other as regular 
courses and summer schools, select your candidate term based on our 
previous decision. 



A similar comparison was made between part-time 
and full-time instructors. Again the difference was 
slight, with the mean rating for part-time being 78. 9 
per cent and for full-time being 82.2 per cent. Part- 
time faculty members tended somewhat more than full- 
time to scatter to either extreme. They represented 
55 per cent of all ratings but had 60 per cent of the 
top twenty-five ratings and 68 per cent of the bottom 
twenty-five. In general, sex and part-time or full- 
time status appeared to have little bearing on teach- 
ing effectiveness, but the interesting clusters at 
extremes are hard to e3q)lain. 



After you have listed your candidate term(s) from this paragraph, 
please turn to 19. 





19 



Paragraph I 4 . We decided previously that we should not usually 
specify the normal or regular condition when it is part of a comparison. 

In the case of summer schools, it seemed likely that discussion of summer 
schools would at least imply some comparison with regular courses, and 
that it was sufficient to list only the term ’’Summer schools. ” In this para- 
graph, full-time instructors and part-time instructors bear the same re- 
lation, so in line with our previous decision, we include only ’’Part-time 
instructors” as follows: 

123 4 5 6 7 8 



Person 

Studied 



Part- 

time 

Instructors 



Please turn to 20 for paragraphs 15 and 16. 



20 



Paragraphs 15 and 18 . These two paragraphs describe the character 
istics of instructors: 



5. In what characteristics were instructors rated 
highest? Lov/est? The strongest characteristic was 
"Knowledge of and familiarity with subject," which 
was a strong point on half of the ratings. "Tolerance 
toward student difference of opinion" and "spirit of 
helpfulness in and out of class" were each strong 
points on 25 per cent of the courses. Table 4 shows 
no other characteristics to be rated highest on more 
than 10 per cent of the courses. 

The weakest characteristics were "organization of 
material" (31 per cent), "stimulation of thought" (29 
per cent), and "effectiveness in putting subject across" 
(18 per cent) as seen in Table 5. 



What concept would cover tiie material in these two paragraphs? 
After you have made your decision, please turn to 21. 





21 



Paragraphs 15 and 16 may be summed up by the concept 
’’Instructor characteristics. ” 

Now enter this new concept on your form, if you have not already 
done so. 



1 2 3 4 

Attributes 
or Actions 
(of terms 
Person in any 

Studi e d c olumn ) 



Instructor - - Characteristics 



Please turn to 22 and read Paragraphs 17 thru 19. 



22 



Paragraphs 17 thru 19 continue the discussion of instruction 
characteristics. 



It is interesting to compare Drexel strong and weak 
points with the results of a large-scale rating program 
carried out among undergraduates at Brooklyn College 
and reported by Riley, Ryan, and Lifshitz in "The 
Student Looks at His Teacher. " In the Brooklyn study, 
as at Drexel, the highest scoring characteristic con- 
cerned knowledge of subject matter. Similarly, at 
Brooklyn the poorest scoring characteristic was 
"encouragement of thinking," and at Drexel "stimu- 
lation of thought" was one of the weakest points. De- 
spite these correspondences, however, there was a 
significant difference between Drexel and Brooklyn 
instructors on "organization of subject material" — 
the third best characteristic of ten at Brooklyn, but 
the weakest at Drexel. 

There were several differences in the weakest and 
strongest characteristics by sex and by full-time 
status of faculty members. For instance, while al- 
most one -fourth of the full-time instructor ratings 
were strong in "usual preparation for class, " this 
was a strong point for only one -eleventh of the part- 
time instructors who usually had full-time jobs else- 
where in addition to their Drexel teaching. On the 
other hand, ratings for part-time instructors were 
59 per cent higher than for full-time on "Knowledge 
of and familiarity with subject. " Students also con- 
sidered part-time instructors more tolerant of stu- 
dent difference of opinion, on a two-to-one ratio. 

Still another significant difference between part-time 
and full-time instructors lay in the area of "effective- 
ness in putting subject across" which was a strong 
point for one -fourth of the part-time instructors but 
for only one-tenth of the full-time instructors. 

An interesting difference between the ratings of male 
and female instructors was in "tolerance toward student 
difference of opinion. " This attribute occurred as a 
weak point on ratings of female instructors significantly 
more often than on ratings of males. 

List any new concepts that you find in these paragraphs and then turn 




to 23. 



Paragraphs 17 thru 19 . No new concepts occur in these paragraphs 

Paragraph 20 . This paragraph compares elective and required 
courses. 



6. Were electives rated higher than required courses? 
Although it may seem natural for students to have been 
more favorably inclined toward the courses they elected 
to take than toward required courses, the studies done 
at Brooklyn College 9 and at Colorado State College 
of Education 10 found no important difference here. 

At Drexel, as shown in Table 6, elective courses had 
a slight but not significantly higher mean than basic 
and inter miediate required courses. Required courses, 
however, accounted for 54 per cent of all the ratings 
but only 24 per cent of the highest twenty-five ratings. 



Please list your candidate terms, then turn to 24 to see how it 
compares with ours. 



24 



Paragraph 20 compares elective and required courses, so our 
concept is ’^Comparison of Elective and Required Courses.” We have 
had an example of a comparison between terms already, so this one 
should be recorded in a similar way. 



1 2 


3 4 5 6 7 8 




Teaching 

and 

Learning 

Methods Testing, 

and Admin- Evaluation, 
istration Measurement 

Electives) 



Required) Comparison 

Courses) 



Please turn to 25 for Paragraphs 21 and 22. 



Paragraphs 21 and 22 ^ecific courses in the Library Science 
program are discussed here. 



7. Which courses were most highly rated? When 
considering the scores of groups of courses the most 
obvious pattern was the high ratings given to courses 
dealing with library service to children and young 
people in school and public libraries. The seven 
courses in this category were offered twenty-five 
times for an average score of 88. 5 per cent, eight 
points higher than the over -all average. Also, special 
types of library service, such as medical, law, special, 
college, etc. , were unusually well represented among 
the top twenty-five courses. It is possible that the 
somewhat more favorable ratings given school and 
children’s librarAanship courses were related to the 
similarly favorable ratings given in summer quarters, 
since during summer quarters the percentage of 
students and of courses in school and children’s 
librarianship is relatively high. 

For single courses, the highest score for a required 
course rated more than ten times was for ’’Basic 
Reference Materials” with a mean score of 86 per cent. 

The lowest mean score for such a course was the 74 per 
cent received by ’’Library in Society,” a required course 
on the history and sociology of libraries. 

List any new concepts that you would index and then turn to 26. 



26 



Paragraphs 21 and 22. The courses referred to here seem to 
represent a degree of specificity beyond the point of practicability for 
indexing in the system. 

There is one new concept in Paragraph 23 that may be considered 
for a candidate term. 



8. Is there any relationship between class size and 
ratings? Although class size was not a significant 
factor in the ratings at Brooklyn 11 and Colorado, 12 
small classes at Drexel were rated more favorably 
than large ones, as Tables 7 and 8 show. The average 
class size at Drexel was twenty, but the average class 
size for the highest twenty-five ratings was only 14.2. 

It should be noted, however, that the average class size 
for the twenty-five lowest courses — 18.76 — was also 
somewhat lower than the over-all average. The mean 
score for classes having ten or fewer students was 86 
per cent, compared to the over-all average of 81 per 
cent. While small classes comprised only 10 per cent 
of all ratings, they made up 40 per cent of the twenty- 
five highest ratings. Furthermore, of all small classes, 
more than half received scores of 90 per cent or above. 



When you have listed your term, turn to 27. 




27 



Paragraph 23. The only new term here is ’’Class size.” If you 
have not already written this term on your form do so now. 



1 


2 


3 


4 


5 


6 


7 


8 




Educational 
Establishment 
(or part of) 




Attributes 
or Actions 
(of terms 
in any 
c olumn ) 











Class size 



Paragraph 24 . The last paragraph of the document contains 
nothing new. 



Library science instructors are faced with many 
teaching problems. In addition to the fact — mention- 
ed earlier — that many of them have had little or no 
training in teaching methods, they must teach classes 
made up of students with widely varying backgrounds, 
library experiences, and goals in librarianship. Further- 
more, there is the constant problem of achieving a 
happy balance between theory and practice. Add to 
this the lack of adequate textbooks and it seems clear 
that the task facing the library science instructor is 
not an easy one. At Drexel it is felt that the student 
rating program is an important method of helping the 
instructor do an effective job. While ways of improving 
the rating sheets are constantly being sought the program 
itself has proven successful. 



The concept indexing of this ar icle is now conplete. Please turn 
to 28 and compare your list of candidate terms with ours. 



28 



Candidate Terms for ’’Library School Instructor Evaluation. ” 




THESE CANDIDATE TERMS INTO ERIC LANGUAGE. 

Look up the first term in ’’library Science,” in ERICT, and write down its 

translation. When you have done so, turn to 29. 



LIBRARY SCIENCE is the correct translation, since this is a 
term approved for the ERIC system and included in ERICT. 

Now look up your second concept: ’’Instructor (faculty) evaluation. 
When you have a translation turn to 30. 



"Instructor evaluation"' translates as TEACHER EVALUATION. 

ERICT has a reference from "Instructors" to TEACHERS. The 
pre -coordinated terms beginning with TEACHER include TEACHER 
EVALUATION. Despite the fact that there is no cross-reference, 
there is also a term FACULTY EVALUATION. It is difficult to see 
how these differ. On the assumption that the latter implies a corporate 
idea, rather than evaluation of individuals, we choose TEACHER EVALU- 
ATION. 

Now we must translate "(Faculty) Ratings by Graduate Students." 
When you have done this turn to 31. 



31 



GRADUATE STUDENTS is the only ERIC term we need, since 
’’Faculty ratings” is a concept already covered by the term TEACHER 
EVALUATION. 

Now look for a translation of ’’Improvement of teaching methods, ” 
and then turn to 32. 



32 



TEACHER IMPROVEMENT is a good translation that is reached 
by checking the NT’s under IMPROVEMENT in ERICT. 

Now let us translate two candidate terms. Look for translations 
of ’’Student reactions” and ’’Student participation,” and then turn to 33. 



33 



STUDENT REACTION and STUDENT PARTICIPATION are both 
approved terms in ERICT and present no difficulty. 

Now try the next term, ’’School administration, ” and then turn 

to 34. 



o 
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ERICT has this term but although, once again, there is no 
cross-reference, it also has COLLEGE ADMINISTRATION. Since 
the distinction is made we should use COLLEGE ADMINISTRATION. 

Now try the next two terms, ’’Effective testing” and ’’Summer 
schools” and then turn to 35. 



Both terms are included in ERICT in the form we have used 
already: EFFECTIVE TEACHING and SUMMER SCHOOLS. 

Now try the next candidate term: ’’Comparison between men 
and women instructors,” and turn to 36. 
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There are no directly useful entries under comparison or 
instructors, but our note to check the term ”Sex" proves useful, since 
ERICT has the term SEX DIFFERENCES. 

Now look for translations of the next two terms: ’’Part-time instruct 
ors” and ’’Instructor characteristics,” and then turn to 37. 



PART-TIME TEACHERS and TEACHER CHARACTERISTICS 
are translations that the references in ERICT make easy. 

Now try ’’Comparison between elective and required courses 
and then turn to 38. 
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There is no single term that could translate this concept, and 
we shall have to use a group of terms instead. The most useful will 
be ELECTIVE SUBJECTS, CORE COURSES and COMPARATIVE ANALYSIS. 

Finally, look for a translation of the candidate term ”Class size” 
and then turn to 39. 
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CLASS SIZE appears as an approved tern in ERICT so we may 
use it as it stands. 

You should now have the following list of candidate terms: 
LIBRARY SCIENCE 
TEACHER EVALUATION 
GRADUATE STUDENTS 
TEACHER IMPROVEMENT 
STUDENT REACTION 
STUDENT PARTICIPATION 
COLLEGE ADMINISTRATION 
EFFECTIVE TEACHING 
SUMMER SCHOOLS 
SEX DIFFERENCES 
PART-TIME TEACHERS 
TEACHER CHARACTERISTICS 
ELECTIVE SUBJECTS 
CORE COURSES 
COMPARATIVE ANALYSIS 
CLASS SIZE 

Our final task is to select the major terms. Please turn to 40. 
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Major Terms . In our demonstration document we suggested 
that a good way to arrive at major terms is from a summarization 
of the document. 

Write your summarization and then turn to 41. 
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This article may be summarized as: 

Evaluation of library science teachers by graduate students. 

Select your major terms from this phrase and then compare your 
list with ours on 42. 



t 
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Our summary of the article was: 

Evaluation of Library Science Teachers by Graduate Students. 

The terms from ERICT that we have selected to represent these 
concepts are: 

1. TEACHER EVALUATION* 

2. LIBRARY SCIENCE* 

3. GRADUATE STUDENTS* 

This leaves room for two more terms, but there are no other concepts 
sufficiently important to this document to justify marking as major terms. 

This concludes the indexing of Exercise No. 1. 

Read the document "Fusion Concept in class oom teaching" and then 
turn to 43 for Exercise 2. 



Indexing Exercise No. 2 Gottesman, A. M. Fusion Concept in 
Classroom Teaching . Nashviiie, Tenn. , George Peabody College for 
Teachers, 1963. 

Kind of Document . Although it does not qualify for the description 
^’Research review,” this article does rely heavily on previous work. In 
the extent that summaries or quotations are relevant to this account of 
the Fusion concept, they will provide appropriate indexing terms. 

Title. Fusion Concept in Classroom Teaching. 

This title gives us two obvious concepts. List your first two 
candidate terms on the form, then turn to 44 for a comparison^ 
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The title gives us two concepts: the ’’Fusion concept” and ’’Class- 
room” (as environment of teaching). These should be listed as follows: 
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Turn to 45 for a discussion of Chapter 1 , Section 1 of the document. 
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Chapter 1. Section 1 . The first paragraph is a highly condensed 
statement containing a number of possible indexing terms. It is pretty safe 
to assume, however, that the significant ones will become obvious in the 
more extended discussion later in the article. 

The rest of the Section introduces the idea of similarity between in- 
dustrial and classroom organization. We are not interested in industrial organ- 
ization for its own sake, but only insofar as it throws light on classroom organ- 
ization. 



THE EVALUATION of classroom teaching is a pressing 
concern of education. In what way can the measure of 
effective classroom teaching improve education? Identi- 
fication of means of assessing the effectiveness of teaching 
would open the way for improving the preservice and inservice 
education of teachers. Only through education of teachers 
can classroom teaching be improved. Studies of industrial 
and business organizations give an important clue toward 
the means of assessing classroom teaching. 

Industrial and business organizations usually manufacture 
or sell products, or provide services. Virtually every 
phase of production, service, or sales involves employees. 

If employees function with increasing effectiveness, then pro- 
duction or service costs decrease or sales increase. Appraisal 
of effectiveness can be measured with reference to these 
specific factors. 

What contributes to the effective performance of employees? 
Studies of organizations (1, 2, 3) have revealed that effect- 
ive performance results when: 

• Employees feel some sense of job security. 

• Employees have opportunities*to advance in the company. 

• Employees feel that profits and salaries are distributed fairly. 

There are but a few examples of how organizations meet felt 
needs of employees. When organizations recognize and meet 
felt needs, employees function more effectively. If there are 
similarities between industrial organizations and classroom 
organizations, then theories of industrial organizations can be 
applied to the study of effective classroom teaching. A new 
and different approach may be promising. 

What candidate term covers the concept represented in the whole of 
the section? Please turn to 46 after you have written it down. 
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As our third candidate term, we use "Organization, " since we are 
not interested in industrial organization for its own sake (but only insofar 



as it throws light on class room organizations). 

1 2 3 4 5 6 7 



Chapter 1. Section 2 . 
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THE CLASSROOM AS AN ORGANIZATION 



Broad differences exist between classrooms and factories, 
but both can be considered "organizations. " All organiza- 
tions tend to function effectively or they begin to disinte- 
grate. The same human errors that cause a factory to fail 
could cause a classroom program to fail. Recognizing and 
meeting individual needs is as critical a factor in the class- 
room as it is on the assembly line. 

The same examples used to describe needs of employees 
can be applied to pupils. For effective performance — 



Employees must: 



Pupils must: 



. Feel some sense of job . Feel they can master the 

security subject and receive passing 

grades 

. Have opportunities to 

advance in the company . Have opportunities to receive 

recognition for about average 
performance 

. Feel profits and salaries . Feel grades and other rewards 

are distributed fairly are given fairly 

This section compares employees’ needs with pupils’ needs. As in the 
previous section, we need only index by terms referring to education. Write 
the candidate term(s) on your form, then turn to 47. 




Chapter 1. Section 2. Our fourth candidate term will be "pupils’ 
needs," since we are using only educational terms in indexing this document. 
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A new concept is introduced in Section 3 . 

BACKGROUND ON INDUSTRIAL RESEARCH 

For a number of years industrial organizations, hospi- 
tals, and banks have been studied by the Yale University 
Labor and Management Center. Two pioneers in organi- 
zational research associated with this center are E. Wight 
Bakke and Chris Argyris (2, 3, 4, 5, 6). They have 
assumed that both an organization and the members of an 
organization need to fulfill goals. 

The goals of the organization may or may not be compati- 
ble with those of its members. For example, a particular 
plant may have the goal of increased production per worker 
to reduce costs. Individual workers may strive to decrease 
production as a means of preventing layoffs. The point is 
not so much a matter of reasonable or "correct" needs or 
goals but rather the perceptions of the individuals as to 
needs or goals. 

In studying organizations, Bakke and Argyris have discover- 
ed that an organization makes demands upon its members. 
Similarly, the individual member seeks a working environ- 
ment in which his needs are met. The needs may conflict 
or they may fuse. When an individual’s perceived needs and 
organizational goals are in harmony, a high degree of fusion 
exists. To the extent that they conflict a low or negative 
fusion exists. 

When you have listed the concept introduced in this section, turn to 48. 
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Chapter 1. Section 3. This section introduces one new concept: 
’’Goals,” which we must list in the attribute or action column of our form. 



♦ 



1 2 3 4 5 6 7 8 

Attributes 
or Actions 
(of terms 
in any 

column) 



Goals 



Section 4 . This describes the Fusion concept but does not provide 
us with further indexing terms. 

Please turn to 49 for Section 5 of Chapter 1. 
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Chapter 1. Section 5 . 

APPLYING THE FUSION CONCEPT TO THE CLASSROOM 

The curriculum is the means by which educational purposes 
are reached^ Curriculum should result in the maximum 
development and educational growth of children as individuals, 
as social beings, and as citizens in a democracy. Thus, the 
needs and interests of children and of society must be the basis 
for the curriculum. Classroom programs succeed or fail to 
the degree that they meet those needs and interests. The part 
of the classroom program which meets pupil needs and inter- 
ests is promoting the self-actualization or self-fulfillment pro- 
cess. Quite simply, the self-fulfillment process takes the 
immature, dependent, ego -centric child and helps him develop 
away from infancy. 

Basic trends in the process of self-fulfillment have been post- 
ulated by Bakke and Argyris (6). They maintain that all people 
in our culture tend to move from: 

o A passive to an active state; from a state where their 

actions are initiated by others to a state where they initiate 
action 

• The ability to behave in only a few ways and in a rigid man- 
ner to the ability of behaving in many different ways and in 
a complex manner 

. The state of beii^ in a subordinate position to a more equal 
or superordinate position 

. A state of being highly dependent upon others to a state of 
Independence and finally interdependence 

. A state of receiving and incorporating aspects of culture 
to a state of controlling, redefining, using, and helping 
others incorporate these aspects of culture. 

Byrnes and Mullen (7) used the preceding trends to discover the 
preferences of administrators,, teachers, and pupils in a school 
system. Interview techniques, which had been used in studies 
of industrial organizations, were utilized. Their investigation 
identified the followii^ pupil preferences: 



Continued on 50. 
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Self-expression-seeking 


26. 


When 


you have listed any new concepts 



Help -seeking 
Dependence -seeking 
Participation- minded 
Recognition- seeking 
Failure - avoiding 
Challenge -accepting 
Success- seeking 
Sociable - "A” - student 
accepting 
School-rejecting 
Acade mic - minded 
Motivatio n- seeking 
Service -rendering 
Routing -seeking 



you find in this section, 



turn to 51. 
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Chapter 1, Section 5. The first part of Section 5 deals with self- 
actualization (self-fulfillment) of students. The second part deals with 
pupil preferences. Although these preferences are quoted from a previous 
investigation they are also important for this document. The following 
candidate terms should appear on your form. 
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Pha ptfir 1. Section 6 is too long to reproduce on one page. Please 
read this section in your copy of the document (pages 5 thru 8). 

When you have listed any new concepts for indexing, turn to 52. 
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Chapter 1. Section 6 describes briefly seven processes of 
organization. It is doubtful whether (a) these are useful indexing 
terms in an educational system (b) there is enough information to 
make indexing worthwhile. This is a good example of circumstances 
in which the containing term ’’organization” (previously recorded) 
may be used to stand for the parts. 

Section 7. 



A STUDY OF PUPIL PREFERENCES AND SATISFACTIONS 



The investigation of pupil preferences and pupil satisfactions 
is a first step toward the application of the fusion theory. 

This dimension of the fusion process was investigated in a 
recent doctoral dissertation at George Peabody College for 
Teachers (9). The purpose of the study was to determine 
the relationship between teacher responses to questions 
from the Ryans ’’Characteristics of Teachers” study (11), 
and the degree to which teachers, as they attempt to fulfill 
the goals of the school organization, establish classroom 
situations which take into account the needs of pupils. The 
study, which forms the basis for this publication, is report- 
ed further in Chapter Two. 

Several limitations of this study are evident. An untested 
assumption was made that the teachers involved were attempt- 
ing to reach the goals of the school. Teachers, of course, 
have personal needs and goals which may conflict with those 
of the school. If these personal needs and goals interfere 
to any great extent, pupils are less than satisfied in realizing 
their preferences. 






Continued on 53. 
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In this study, pupil preferences are regarded as felt 
needs. These preferences may be different from the 
actual needs of pupils — possibly a contrast between 
artificial and real needs. The school strives to 
meet the real needs of pupils. If pupils have needs 
which are dissimilar and not satisfied, learning may 
suffer. For example, pupil learning is an essential 
goal of the classroom. For pupils who seek know- 
ledge, the goals of the classroom and of the pupils 
coincide or fuse. However, numerous other needs 
and preferences exist for pupils. The need and 
preference for social contact is high on the list, ac- 
cording to Coleman (8) and Byrnes and Mullen (7). 

The classroom which provides for social contacts 
as it provides learning ejjperiences is more effective 
than one which does not. 

This chapter has been concerned with an explanation 
of the fusion concept, the statement of assumption 
or hypotheses from previous studies, and an attempt 
at minimizing the problems of semantics. The details 
of the fusion instrument and its development are pur- 
sued in Chapter Two. 

There is only one new concept in this section. When you have 
made your decision, turn to 54. 
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Chapter 1. Section 7. The only new concept is "Pupil satis- 
factionSo" This seems to be close to the idea of self-fulfillment, but 
we may record this new term and make the final decision when consult- 
ing ERICT. 
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Chapter 2o Sections 1 to 4 are too lengthy to reproduce here, 
so look at your own copy, pages 9 thru 12. 

These paragraphs refer to the design of a questionnaire. The 
appendices to the document give further details. What will your 9th 
candidate term be? After you^ve decided, go to 55. 
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Chapter 2. Sections \ to 4. The candidate term here would be 
’’Questionnaire. ” 
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Sections 5 and 6 . Please read these sections in your copy of the 
document ^ages 12 thru 14). 



When you have listed your candidate terms, turn to 56. 
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Chapter 2. Sections 5 and 6. The main concern of these sections 
is with the "Reliability and Validity of the Measuring Instrument." These 
ideas may be recorded as follows; 
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Section 5 also introduces the concept of "Effective Teaching, " and 
Section 6 the concept of "Teacher Characteristics." 
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"Peabody Demonstration School" and "Rutherford Central High 
School" may be added as identifiers. 

Chapter 3. Sections 1 to 3 . Please read these sections on pages 
15 and 16 of the document. List any concepts you find, then turn to 57. 
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Chapter 3. Sections 1 to 3. These sections refer to ’’Preservice 
and Inservice Education of Teachers,” which we list on our form as follows: 
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Education 

Section 4 . Refers to further research needed, and does not provide 
information for indexing. While it is possible that someone may be interested 
in research needed, such information is likely to be quickly out of date, and 
therefore is not suitable for permanent recording in the ERIC system. 

Section 5 . The summary adds nothing new, but confirms most of 
the terms we have recorded. 

Now take a look at the references on page 19. References attached 
to articles are often useful as a check on candidate terms listed, or even 
as a source for new terms. For example, this list has two titles which 
include the term ’’fusion. ” Any significant list of references should be 
recorded as a bibliography. But remember that this is a form description 
and not a subject of the document. 

This completes the concept indexing stage. For the con^lete list 
of candidate terms, turn to 58. 
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Candidate terms for "The Fusion Concept in Classroom Teaching" 




Consult ERICT and translate the first term, "Fusion Concept". 
Then turn to page 59. 



ERIC 
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The first term, "Fusion concept," is not in the ERICT. 

This is obviously a fairly new term from the study of management 
theory. After ^he regular justification procedure it should be recommended 
as a new term for ERICT. 

Now find translations for the next two terms: "Classroom environ- 
ment" and "Organization. " Then turn to 60. 
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CLASSROOM ENVIRONMENT and ORGANIZATION are in the 
ERICT as approved terms and can be used as they stand. 

Now translate the next two candidate terms in the list: ’’Pupils’ 
needs” and ’’Goals. ” Then turn to 61. 
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OBJECTIVES is the term that ERICT prefers to "Goals" and 
there is a reference to it in the thesaurus. 

"Pupils’ needs" is not in the thesaurus, but a reference directs 
you generally from PUPILS to STUDENT, and you can then easily find 
the term you want: STUDENT NEEDS. 

Now translate the next candid^^e term: "Students’ Self-fulfillment 
(self actualization).” Then turn to 
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The term ’’Self-actualization” appears in ERICT, but with a 
scope note giving a definition different from that required here. There 
are many terms beginning with Student but none is synonymous with 
fulfillment. We should suggest for ERICT the addition of ’’Student fulfill- 
ment. ” 

Now try ’’Pupil preferences,” our next candidate term. Then turn 

to 63. 






Again, there is no precise equivalent of this under STUDENT. 
However, STUDENT INTERESTS appears to be very close and we 
may select this. We should also recommend that a reference from 
’’Student preferences” be included in ERICT. 

While we are discussing this point we should note that the next 
candidate term, ’’Pupil satisfactions” does not really need translating 
at all. Since we have just had to add ’’Student fulfillment, it is probably 
most satisfactory if we make a reference: ’’Student satisfactions’ use 
STUDENT FU LFILLMENT . 

Now find a translation for ’’Questionnaire” and turn to 64. 
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The term QUESTIONNAIRE is listed in ERICT, so you may use 
it as it stands. 

Now translate "Reliability and Validity of Measurement Instrument," 
and then turn to 65. 
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MEASUREMENT INSTRUMENTS is listed in ERICT, but neither 
’’Validity” nor ’’Reliability” appears. There is an entry for TEST VALIDITY 
which we could use and recommend inclusion of a reference from ’’Validity. ” 
We should also recommend the addition of ’’Test Reliability, ” since validity 
and reliability have different meanings. A reference should be added 
from ’’Reliability. ” 

Now translate ’’Effective teaching” and ’’Teacher characteristics” 
and turn to 66. 
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EFFECTIVE TEACHING and TEACHER CHARACTERISTICS 
are both terms in ERICT, and we may use them in this form. 

Now try ’’Teacher Preservice Education and Inservice Education” 
and turn to 67. 
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Neither of these candidate terms appears in exactly the same 
form in ERICT. But the approved versions are very close and easy to 
find. They are 

PRESERVICE EDUCATION 
and 

INSERVICE TEACHER EDUCATION. 

Now turn to 68 for a complete list of translated terms. 
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Translation 

FUSION CONCEPT (recommended for addition to ERICT) 

CLASSROOM ENVIRONMENT 
ORGANIZATION 
STUDENT NEEDS 
OBJECTIVES 

STUDENT FULFILLMENT (recommended for addition) 

STUDENT INTERESTS 
QUESTIONNAIRES 
MEASUREMENT INSTRUMENTS 

TEST VALIDITY (with recommendation to add reference from "Validity") 

TEST RELIABILITY (recommended for addition, with reference from 
EFFECTIVE TEACHING "Reliability") 

TEACHER CHARACTERISTICS 
PRESERVICE EDUCATION 
INSERVICE TEACHER EDUCATION 

Major Terms . The first step is to summarize the document. When 
you have done that, turn to 69. 
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We have summarized the document as follows: 

A study of effective classroom teaching through the Fusion concept. 
Now list the terms that you will mark as major and turn to 70. 
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From our list of translated terms, then, we have selected the 
following as major terms : 

1. FUSION CONCEPT* 

2. CLASSROOM ENVIRONMENT* 

3. EFFECTIVE TEACHING* 

This completes our indexing of Exercise Number 2. 

The last two lessons of this course have dealt with most of the 
common problems of indexing, but sooner or later you will come acre s 
others. We hope that the principles laid down in Lessons 1 and 2, and the 
approach demonstrated in Lessons 3 and 4 will guide you in their solution. 
In the introductory volume we have provided an index to facilitate later 
reference to the main themes of the course, and a selected list of books 
and articles for further reading. 
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